Optimal threshold analysis of segmentation methods for identifying target customers

In CRM (Customer Relationship Management), the importance of a segmentation method for identifying good customers has been increasing. For evaluation of different segmentation methods, Accuracy often plays a key role. This indicator, however, cannot distinguish the following two types of errors: Type I Error for misidentifying a good customer as a bad customer and Type II Error for misinterpreting a bad customer as a good customer. In order to analyze the financial effectiveness of various segmentation methods, it is crucial to capture the distinction between Type I and Type II Errors since the former represents the opportunity cost while the latter results in the inefficient use of the promotion budget. The purpose of this paper is to overcome this pitfall by introducing two different indicators: Recall and Precision, which have been prevalent in the area of Information Retrieval. A mathematical model is developed for describing a generic segmentation method. Assuming that a promotion is addressed exclusively to the selected target customers, the financial effectiveness of the underlying segmentation method is expressed as a function of Recall and Precision. An optimization problem is then formulated so as to maximize the financial measure by finding the optimal threshold level in terms of the severeness for estimating the target set of good customers. By introducing a functional form which represents correctness and mistakes about the target set, the unique optimal solution is derived explicitly. Using real customer purchase data, the proposed approach is validated where Logistic Regression Model and Support Vector Machine are employed as segmentation methods. The methodology developed in this paper may provide a foundation for understanding and comparing the performance characteristics of various segmentation methods from a new perspective.